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IN THE CLAIMS : 

1 1 . A method for interpreting nonverbal communicative behavior of an 

2 individual, the method comprising the steps of: 

3 (a) observing and recording a behavior performed by the individual; 

4 (b) acquiring access to a collection of behavior codes, each behavior code 

5 having an associated explanation; 

6 (c) assigning an observed behavior code to the observed behavior; 

O 7 (d) comparing the observed behavior code to the collection of behavior 

'?1 8 codes; 

(e) retrieving the explanation associated with each behavior code of the 
jlO collection which matches the observed behavior code. 

:: 

1 2. The method according to Claim 1 , further comprising the step of storing 

Q2 the observed behavior. 

1 3. The method according to claim 2, wherein the recorded behavior is stored 

2 in segments. 



4. The method according to Claim 3, wherein a selected segment is 
transmitted by a user via a user interface to a computer, wherein the computer assigns the 
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observed behavior code, searches the collection of behavior codes for matches by 
comparing the observed behavior code to the collection of behavior codes, and retrieves 
the explanation associated with each behavior code which matches the observed behavior 
code. 

5. The method according to Claim 2, wherein the recording is comprised of a 
string of frames, and the selected segment received by the computer is compressed into a 
compressed file by sampling the frames of the selected segment, and digitizing and 
formatting the sampled frames. 

6. The method according to Claim 5, wherein the recording is further 
comprised of a string of audio frames and image frames. 

7. The method according to Claim 5, wherein recorded inactivity is filtered 
out and not included in the digital compressed file. 

8. The method according to Claim 1, wherein the step of assigning an 
observed behavior code to the observed behavior further comprises: 

recognizing behavior units which comprise the behavior; 
assigning a behavior unit code to each behavior unit; 
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combining the behavior unit codes to form an observed behavior code, 

9. The method according to Claim 8, wherein the behavior units comprise 
gesture units and sound units; and wherein the behavior unit codes comprise gesture unit 
codes and sound unit codes. 

10. The method according to Claim 9, wherein the gesture unit codes and 
sound unit codes are synchronized to form the observed behavior code. 

1 1 . The method according to Claim 9, wherein the behavior unit codes are 
concatenated in chronological order, and each behavior unit code indicating when the 
associated behavior unit occurred simultaneously with the behavior unit associated with 
the preceding behavior unit code. 

12. The method according to Claim 1 , wherein when a match is not found for 
the observed behavior code, the observed behavior code is added to the collection of 
stored behavior codes as a new code. 

13. The method according to Claim 12, wherein the collection of stored 
behavior codes is a database. 
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14. The method according to Claim 13, wherein an ID is associated with a 
user and an ID is associated with the individual, and the user has the option of allowing 
the user ID and the individual ID to be entered into the database in association with a new 
code, when a new code is added, and in association with any matching stored behavior 
code in the behavior code database. 

15. The method according to Claim 8, wherein the behavior units comprising 
the observed behavior are prioritized according to one of: chronological order, frequency 
of occurrence in database and degree of dominance. 

16. The method according to Claim 15, wherein the step of comparing the 
observed behavior code to the collection of behavior codes further comprises: searching 
the collection of behavior codes for behavior codes having behavior units matching the 
behavior units of the observed behavior code, in order of the priority of the behavior unit 
codes in the observed behavior code. 

17. The method according to Claim 1 6, wherein the step of comparing the 
observed behavior code to the collection behavior codes further comprises: synchronizing 
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3 the comparison of the gesture units and the sound units of the observed behavior to the 

4 gesture units and the sound units of the stored behaviors. 

1 1 8 . A method for interpreting nonverbal communicative behavior of an 

2 individual, the method comprising the steps of: 

3 observing a behavior pattern comprised of multiple behaviors performed 

4 by the individual; 

Q 5 acquiring access to a collection of behavior pattern codes, each behavior 

-J 6 pattern code comprised of a string of behavior codes; 

J::; 7 assigning a behavior pattern code to the behavior pattern, formed by 

*ri 8 stringing together the codes associated with the multiple behaviors; 
jy ; 9 comparing the observed behavior pattern code to the collection of behavior 

n\0 pattern codes; 

f 1 1 reporting the results of the comparison. 



19. The method according to Claim 1 8, wherein the collection of behavior 
codes and the collection of behavior pattern codes are acquired according to past 
behaviors and behavior patterns performed by the individual. 
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1 20. The method according to Claim 1 8/ wherein the collection of behavior 

2 codes and the collection of behavior pattern codes are acquired according to past 

3 behaviors and behavior patterns performed by multiple individuals. 

1 21 . The method according to Claim 1 8 , wherein the method further comprises 

2 the steps of: recognizing events; assigning an event code to each recognized event; and 
J 3 correlating recognized events to one of behaviors and behavior patterns. 

: 1 22. The method according to claim 21, wherein an event causes an 

; 2 influence upon the behavior of the individual and comprises at least one of: a) presence of 

r, 3 an object, b) movement of an object, c) movement of the individual relative to an object, 

i 4 d) an occurrence in the schedule of the individual, e) an occurrence expected to take place 

} 5 in the schedule of the individual, and f) a detected change in the condition of the 

6 individual. 

1 23 . The method according to Claim 2 1 , wherein the step of correlating 

2 recognized events comprises: 

3 associating event data with each behavior code, wherein the event data 

4 comprises the event code of each event which has been correlated with the behavior 
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associated with the behavior code, the time of the occurrence of the event, the number of 
occurrences of the event; and 

updating the event data associated with the behavior code in the collection 
of behavior codes which matches an observed behavior code upon the recognition of an 
event occurring within a selected time interval preceding and following the performance 
of the behavior. 

24. The method according to Claim 23 , wherein the step of correlating 
recognized events further comprises: 

analyzing event data for a selected behavior code from the collection of 
behavior codes, to determine which event occurrences are related to a behaviors 
represented by the selected behavior code according to the number of occurrences and the 
time interval between the event and the occurrence of the behavior. 

25. The method according to Claim 23 , wherein the step of correlating 
recognized events further comprises: determining which observed behaviors performed 
by an individual occur within a selected time interval of one another repetitively, for 
establishing a behavior pattern; comparing correlated events for each of the observed 
behaviors included in the behavior pattern, and determining which of the correlated 
events are the same. 
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1 26. The method according to Claim 1 8, wherein the method further comprises 

2 the steps of: 

3 recognizing the occurrence of an event which could pose a danger to the 

4 individual; 

5 transmitting an alert upon the recognition of a danger posing event. 

%1 1 27 - The method according to Claim 26, further comprising the steps of: 

H ;2 associating a danger alert to behavior codes in the collection of behavior codes for 

D3 behavior codes associated with behaviors which indicate that an individual is in danger; 

4 u P on comparing observed behavior codes associated with behaviors 

Jv! 5 observed following a danger posing event occurred, retrieving existing danger alerts for 

f":6 each matched behavior code; and 

7 transmitting an alert upon retrieval of a danger alert. 

1 28 . A system for interpreting nonverbal communicative behavior of an 

2 individual, the system comprising: 

3 a database of stored behavior codes, each behavior code having an 

4 associated explanation; 
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means for accessing the database; 

means for recording a behavior performed by the individual; 

means for assigning an observed behavior code to the recorded behavior; 

means for comparing the observed behavior code to the stored behavior 
codes in the database of stored behavior codes; and 

means for retrieving the explanation associated with the stored behavior 
codes which match the observed behavior code. 



29. A system for interpreting nonverbal communicative behavior of an 
individual, the system comprising: 

means for recording a behavior performed by the individual; 

a user interface in communication with the recording means for receiving 
recordings; and 

an Analysis Provider Server in communication with the user interface, said 
Analysis Provider Server receiving a recording of a behavior; 

said Analysis Provider Server comprising; 

means for assigning a recorded behavior code to the recorded behavior; 

a Behavior Code Database for storing known behavior codes associated 
with known behaviors, each known behavior code having an associated explanation 
associated thereto; and 
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1 3 means for comparing the recorded behavior code with the known behavior 

1 4 codes m the Behavior Code Database and retrieving the explanation associated with each 

1 5 of the known behavior codes which match the recorded behavior code. 

1 30. The system according to Claim 29, wherein the received recording is a file 

2 of a series of frames; and wherein 

3 the means for assigning a recorded behavior code to the recorded behavior 
O 4 comprises : a Digitize and Sample Module for sampling the frames of the received 

^ 5 recording at regular intervals and forming the sampled frames into a digital compressed 

fJ!J 6 formatted File of Sampled Frames with each sampled frame having a time stamp 

1~ 7 indicative of the relative time it was recorded. 

HI 1 3 1 . The system according to Claim 30, wherein the Analysis Provider Server 

O 2 further comprises an Individual Personal Data database for storing data about the 

3 individual, the data comprising identification, age, body measurements and voice 

4 characteristics. 

1 32. The system according to Claim 3 1 , wherein the Digitize and Sample 

2 Module is in communication with a Scaling Module for applying a scaling factor selected 

3 according to the data stored in the Individual Personal Data database to the sampled 
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frames of the digital compressed file for providing a standardized version of the sampled 
frames to the means for assigning a recorded behavior code to the recorded behavior. 



1 33. The system according to Claim 30, wherein the Digitize and Sample 

2 Module is in communication with a Noise Elimination Module for eliminating from the 

3 sampled frames recorded activity which is not meaningful to behaviors of the individual. 

1 34. The system according to Claim 30, wherein the means for assigning a code 

2 to the recorded behavior comprises: a Behavior Coder (BC) Module for recognizing 

3 behavior units which comprise the behavior, assigning a behavior unit code to each 

4 behavior unit, and combining the behavior unit codes to form a recorded behavior code. 

5 and wherein the known behavior codes are comprised of behavior units 

6 codes. 

1 35. The system according to Claim 34, wherein the behavior units comprise 

2 gesture units and sound units, and wherein the behavior unit codes comprise gesture unit 

3 codes and sound unit codes. 
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1 36. The system according to Claim 35, wherein the BC Module is in 

2 communication with a Synchronizer Module for synchronizing the gesture unit codes and 

3 the sound unit codes to form the recorded behavior code. 

1 37. The system according to Claim 35, wherein the BC Module is in 

2 communication with a Behavior Begin/End Detector Module for determining the end of 

3 the recorded behavior and the beginning of a subsequent recorded behavior. 

1 1 38. The system according to Claim 35, wherein the BC Module concatenates 

1 2 the behavior unit codes in chronological order, with each behavior unit code indicating 

1 3 when the associated behavior unit occurred simultaneously with the behavior unit 
L 4 associated with the previous behavior unit code. 

) 1 39. The system according to Claim 35, wherein the behavior unit codes are 

2 comprised of property unit codes, wherein the property unit codes of a gesture unit code 

3 comprise a body part unit code, a movement type unit code, a body position unit code and 

4 a duration unit code. 
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1 40. The system according to Claim 39, wherein the property unit codes of a 

2 sound gesture code comprise a sound description unit code, a sound intensity unit code, a 

3 sound pitch unit code and a duration period code. 

1 41. The system according to Claim 39, wherein the behavior unit codes 

2 comprising the recorded behavior code are prioritized according to one of: chronological 

3 order, frequency of occurrence in database and degree of dominance. 

1 42. The system according to Claim 41 , wherein the means for comparing the 

2 recorded behavior code with the known behavior codes stored in the database comprises: 

3 a Behavior Code Matching (BCM) Module for searching the Behavior 

4 Code Database for behavior codes having behavior units having property units matching 

5 the property units of the observed behavior code. 

1 43. The system according to Claim 42, wherein the Behavior Code Database 

2 is searched according to the priority of the behavior unit codes and the property unit 

3 codes. 
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1 44. The system according to Claim 41, wherein the means for comparing the 

2 recorded behavior code with the known behavior codes stored in the database 

3 synchronizes the comparison of the gesture unit codes and the sound unit codes of the 

4 recorded behavior code to the gesture unit codes and the sound unit codes of the stored 

5 behavior codes. 

1 45. The system according to claim 42, wherein a user enters, via the user 

2 interface, a requested series of behavior codes; 

3 and the Analysis Provider Server comprises a Behavior Series Matching 

4 (BCM) Module for determining when the individual performs a series of behaviors which 

5 correspond to the requested series of behavior codes. 

1 46. The system according to claim 45, wherein the Analysis Provider Server 

2 further comprises a Web Server for receiving data from and transmitting data to the user 

3 interface. 

1 47. The system according to claim 46, wherein the Analysis Provider Server 

2 further comprises a Subscriber Database for storing data relative to the users of the 

3 system comprising an identification number, personal data and financial data. 
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1 48. The system according to claim 47, wherein the Behavior Code Database 

2 comprises a record for each behavior code, each record comprising a field for storing: 

3 the behavior code; an Identification Number for the behavior code; a link 

4 to the file of the received recording; a link to the File of the Sampled Frames; a primary 

5 explanation for the behavior code; a link to a list of similar behavior codes; a link to a list 

6 of related explanations for the behavior code with dates of entry and identification 

7 numbers for the submitting users; recommendations related to the behavior code with 
, 8 dates of entry and identification numbers for the submitting users; an identification 

) 9 number for the user who first submitted the behavior code; an identification number for 

1° the individual; and an indicator that the behavior code indicates that the individual is in 

1 1 danger. 

s ." 1 49. The system according to claim 48, wherein the Analysis Provider Server 

= 2 further comprises an Individual Behavior Codes Database for storing records for at least 

3 one of behavior codes or series of behavior codes performed by the individual, wherein 

4 each record comprises a field for storing: 

5 the behavior code; a key number for the behavior code; a link to the file of 

6 the received recording; a link to the File of the Sampled Frames; a primary explanation 

7 for the behavior code; a link to a list of similar behavior codes; a link to a list of related 
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explanations for the behavior code; recommendations related to the behavior code; and an 
indicator that the behavior code indicates that the individual is in danger. 



1 50. The system according to claim 49, wherein the Analysis Provider Server 

2 further comprises an Individual Event Database for storing events relating to the 

3 schedule, environment and condition of the individual, each event having an event code 

4 and a danger indicator which indicates if the event poses a danger to the individual. 

1 51. The system according to claim 50, wherein the Analysis Provider Server 

2 further comprises a Situational Analyzer (SA) Module for correlating events to recorded 

3 and stored behavior codes and series of behavior codes; 

4 wherein the events influence the behavior of the individual and comprise 

5 at least one of: a) a presence of an object, b) movement of an object, c) movement of the 

6 individual relative to an object d) an occurrence in the schedule of the individual, e) an 

7 occurrence expected to take place in the schedule of the individual, or f) a detected 

8 change in the condition of the individual. 

1 52. The system according to claim 5 1 , wherein the Situational Analyzer 

2 Module comprises: 

3 a General Event Database; 
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4 an Event Identifier/Tracker (EI/T) Module for tracking objects and 

5 recognizing the occurrence of events in the File of Sampled Frames; and 

6 an Event/Behavior Correlator (E/BC) Module for correlating recognized 

7 events with behaviors recorded in the File of Sampled Frames. 

1 53. The system according to claim 52, wherein the EI/T Module recognizes 

2 objects in the File of Sampled Frames and identifies the objects by looking them up in the 

3 Individual Event Database and the General Event Database. 

1 54. The system according to claim 52, wherein the SA Module further 

2 comprises a Labeling Module for identifying events; and 

3 wherein the EI/T Module queries the Labeling Module when a recognized 

4 event is not identified, and upon receiving a successful identification of the event from 

5 the Labeling Module, the EI/T adds the event into at least one of the Individual Event 

6 Database or General Event Database. 



55. The system according to claim 49 wherein each record at least one of the 
Behavior Code Database and the Individual Behavior Code Database further comprises a 
field for storing a link to event data, wherein the event data comprises an event code for 
each event which has been correlated with the behavior code, the time of the occurrence 
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5 of each correlated event relative to the occurrence of the behavior associated with the 

6 behavior code, and the number of occurrences of each correlated event. 

1 56. The system according to claim 52, wherein the E/BC Module correlates 

2 recognized events to the behavior recorded in the File of Sampled Frames by determining 

3 when a recognized event occurs within a selected time interval preceding and following 

4 the performance of the behavior; and 

5 updates the associated event data of the behavior code associated with the 

6 performed behavior in at least one of the Behavior Code Database and the Individual 

7 Behavior Code Database. 

1 57. The system according to claim 52, wherein the Individual Event Database 

2 and the General Event Database include a field for each event which indicates when the 

3 occurrence of the event poses a threat to the individual; 

4 and wherein the S A Module causes an alert to be transmitted to a caretaker 

5 of the individual upon recognition of the occurrence of danger posing event. 
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1 58. The system according to claim 5 1 , wherein the SA Module further 

2 comprises a Behavior Pattern Recognizing (BPR) Module, wherein said BPR Module 

3 recognizes the repetitive occurrences of series of behavior codes for establishing behavior 

4 patterns and recognizes the occurrence of a behavior pattern performed by the individual. 

1 59. The system according to claim 29, wherein the AP Server provides the 

2 user interface with interactive web pages to enable a user 

3 to submit one of a live stream of recordings and a selected recorded 

4 segments; 

5 and to enable the user to request that the AP Server recognize at least one 

6 of a specific behavior, a specific series of behaviors, a specific event, all behaviors and all 

7 repeated behavior pattern. 

1 60. The system according to Claim 59, wherein the AP server enables the user 

2 to request that the AP Server correlate at least one of a behavior to a specified event, an 

3 event to a specified behavior, an event to a specified series of behaviors, and correlation 

4 of events to all recognized behaviors. 
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1 61. The system according to claim 59, wherein the user specifies a behavior 

2 and a series of behaviors by entering, at a Web Page provided, one of the behavior codes 

3 for each behavior, the identification number for each behavior and behavior units 

4 comprising the behavior. 

1 62. The system according to claim 50, wherein the user enters events into the 

2 Individual Event Database, at a Web Page provided, by one of: entering descriptions of 

3 each event; viewing the recording in a view window on a Web Page, selecting an object 

4 displayed in the view window and identifying the object; and viewing the recording in a 

5 view window on a Web Page and entering the event which occurred at the time shown in 

6 the view window. 

1 63. The system according to claim 29, wherein the Analysis Provider Server 

2 further comprises a Query Module for processing user requests to enter and access data 

3 stored in the AP Server. 

1 64. The system according to claim 63, wherein the Analysis Provider Server 

2 further comprises a Report Generator Module for transmitting messages, results from 

3 submitted requests and alerts to the user interface. 
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1 65. A system for interpreting nonverbal communicative behaviors of an 

2 individual comprising: 

3 a camera for recording behaviors of an individual; 

4 a first computer in communication with the camera for selecting a segment 

5 of the recorded behavior; 

6 a second computer in communication with the first computer for pre- 

7 storing a collection of coded definitions of behaviors, each coded definition having an 
H 8 identification code and an explanation; receiving the selected segment from the first 

9 computer; converting said segment into a digital format; sampling said formatted segment 

MO at regular intervals; storing said samples; converting said samples of said formatted 

1 segment into a coded definition of said behavior; comparing said coded definition of said 

: 12 received segment of the recorded behavior to the pre-stored collection of coded 

definitions of behaviors; returning to said first computer the identification code and 

{44 explanation of each of the pre-stored coded definitions which match the coded definition 

15 of the received segment; and if no matches were found; adding the definition code of the 

16 received segment to the collection of pre-stored coded definitions; providing the added 

17 coded definition with an identification code and a blank explanation code; and returning 

1 8 the identification code of the added coded definition to the first computer. 
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1 66. An article of manufacture comprising; 

2 a computer usable medium having computer readable program code means 

3 embodied therein for interpreting a recorded nonverbal communicative behaviors of an 

4 individual, the computer readable program code means in said article of manufacture 

5 comprising: 

6 computer readable program code means for assigning a label to the 

7 recorded behavior; 

O 8 computer readable program code means for accessing a database of 

H 9 behavior labels, each behavior label having an associated explanation; comparing the 

H 0 label for the recorded behavior to the labels in the database of behavior labels; retrieving 

y 1 the explanation associated with each behavior label of the database which matches the 

U 2 recorded label; updating the database by adding the recorded label to the database if a 

J4 3 match was not found; 

rt 4 computer readable program code means for correlating events associated 

1 5 with and included in the recorded behavior to the recorded behavior; and 

1 6 computer readable program code means for providing Graphic User 

1 7 Interface (GUI) for a user to submit recordings to the computer usable medium and 

1 8 request and receive information from the usable medium. 
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1 67. The method according to Claim 2, further comprising the step of: 

2 submitting the stored observed behavior to a service provider, and wherein the service 

3 provider performs steps (b)-(e) for a fee. 
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